AutoML Leaderboard
| Best model | name | model_type | metric_type | metric_value | train_time |
|---|---|---|---|---|---|
| 1_Baseline | Baseline | logloss | 0.652415 | 1 | |
| 2_DecisionTree | Decision Tree | logloss | 0.270943 | 7.57 | |
| 3_Linear | Linear | logloss | 0.149103 | 3.69 | |
| 4_Default_Xgboost | Xgboost | logloss | 0.0853951 | 2.74 | |
| 5_Default_NeuralNetwork | Neural Network | logloss | 0.203507 | 3.81 | |
| 6_Default_RandomForest | Random Forest | logloss | 0.173257 | 5.43 | |
| the best | Ensemble | Ensemble | logloss | 0.0742588 | 0.8 |
AutoML Performance
AutoML Performance Boxplot
Features Importance
Spearman Correlation of Models
Summary of Ensemble
Ensemble structure
| Model | Weight |
|---|---|
| 3_Linear | 1 |
| 4_Default_Xgboost | 5 |
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.0742588 | nan |
| auc | 0.996443 | nan |
| f1 | 0.980488 | 0.482125 |
| accuracy | 0.974803 | 0.482125 |
| precision | 1 | 0.989456 |
| recall | 1 | 0.000187441 |
| mcc | 0.945076 | 0.482125 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.0742588 | nan |
| auc | 0.996443 | nan |
| f1 | 0.980488 | 0.482125 |
| accuracy | 0.974803 | 0.482125 |
| precision | 0.974545 | 0.482125 |
| recall | 0.986503 | 0.482125 |
| mcc | 0.945076 | 0.482125 |
Confusion matrix (at threshold=0.482125)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 434 | 21 |
| Labeled as FALSE POSITIVE | 11 | 804 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 6_Default_RandomForest
Random Forest
- n_jobs: -1
- criterion: gini
- max_features: 0.9
- min_samples_split: 30
- max_depth: 4
- eval_metric_name: logloss
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
4.7 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.173257 | nan |
| auc | 0.98057 | nan |
| f1 | 0.952439 | 0.445515 |
| accuracy | 0.938583 | 0.445515 |
| precision | 1 | 0.984091 |
| recall | 1 | 0.0398722 |
| mcc | 0.86591 | 0.445515 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.173257 | nan |
| auc | 0.98057 | nan |
| f1 | 0.952439 | 0.445515 |
| accuracy | 0.938583 | 0.445515 |
| precision | 0.946667 | 0.445515 |
| recall | 0.958282 | 0.445515 |
| mcc | 0.86591 | 0.445515 |
Confusion matrix (at threshold=0.445515)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 411 | 44 |
| Labeled as FALSE POSITIVE | 34 | 781 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 2_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: gini
- max_depth: 3
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
6.7 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.270943 | nan |
| auc | 0.941587 | nan |
| f1 | 0.933742 | 0.375723 |
| accuracy | 0.914961 | 0.375723 |
| precision | 0.976534 | 0.940678 |
| recall | 0.998773 | 0 |
| mcc | 0.815061 | 0.375723 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.270943 | nan |
| auc | 0.941587 | nan |
| f1 | 0.933742 | 0.375723 |
| accuracy | 0.914961 | 0.375723 |
| precision | 0.933742 | 0.375723 |
| recall | 0.933742 | 0.375723 |
| mcc | 0.815061 | 0.375723 |
Confusion matrix (at threshold=0.375723)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 401 | 54 |
| Labeled as FALSE POSITIVE | 54 | 761 |
Learning curves
Decision Tree
Tree #1
Rules
if (rowid > 3630.5) and (koi_count <= 1.5) and (koi_fwm_stat_sig <= 0.002) then class: FALSE POSITIVE (proba: 98.56%) | based on 1,389 samples
if (rowid <= 3630.5) and (koi_dikco_msky <= 1.755) and (koi_prad <= 13.78) then class: CONFIRMED (proba: 92.69%) | based on 1,204 samples
if (rowid > 3630.5) and (koi_count <= 1.5) and (koi_fwm_stat_sig > 0.002) then class: FALSE POSITIVE (proba: 84.09%) | based on 597 samples
if (rowid <= 3630.5) and (koi_dikco_msky > 1.755) and (koi_count <= 2.5) then class: FALSE POSITIVE (proba: 97.66%) | based on 256 samples
if (rowid > 3630.5) and (koi_count > 1.5) and (koi_dicco_msky <= 1.51) then class: CONFIRMED (proba: 62.43%) | based on 173 samples
if (rowid <= 3630.5) and (koi_dikco_msky <= 1.755) and (koi_prad > 13.78) then class: FALSE POSITIVE (proba: 94.07%) | based on 118 samples
if (rowid > 3630.5) and (koi_count > 1.5) and (koi_dicco_msky > 1.51) then class: FALSE POSITIVE (proba: 93.75%) | based on 64 samples
if (rowid <= 3630.5) and (koi_dikco_msky > 1.755) and (koi_count > 2.5) then class: CONFIRMED (proba: 100.0%) | based on 8 samples
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 3_Linear
Logistic Regression (Linear)
- n_jobs: -1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
2.9 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.149103 | nan |
| auc | 0.985149 | nan |
| f1 | 0.954545 | 0.449334 |
| accuracy | 0.941732 | 0.449334 |
| precision | 1 | 0.999977 |
| recall | 1 | 9.19702e-05 |
| mcc | 0.873411 | 0.449334 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.149103 | nan |
| auc | 0.985149 | nan |
| f1 | 0.954545 | 0.449334 |
| accuracy | 0.941732 | 0.449334 |
| precision | 0.95572 | 0.449334 |
| recall | 0.953374 | 0.449334 |
| mcc | 0.873411 | 0.449334 |
Confusion matrix (at threshold=0.449334)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 419 | 36 |
| Labeled as FALSE POSITIVE | 38 | 777 |
Learning curves
Coefficients
| feature | Learner_1 |
|---|---|
| intercept | 4.82128 |
| koi_depth | 3.93101 |
| koi_impact | 3.85408 |
| koi_dikco_msky | 2.3697 |
| koi_max_sngle_ev | 2.26566 |
| koi_ror | 2.25204 |
| koi_max_mult_ev | 2.07797 |
| koi_dicco_msky | 1.59686 |
| rowid | 1.52785 |
| koi_dor | 1.1722 |
| koi_jmag | 1.06236 |
| koi_ldm_coeff1 | 0.941325 |
| koi_ldm_coeff2 | 0.757603 |
| koi_teq | 0.746536 |
| koi_gmag | 0.618058 |
| koi_fwm_sra | 0.614362 |
| koi_srho | 0.568515 |
| koi_duration | 0.548424 |
| koi_insol | 0.483947 |
| koi_period | 0.476817 |
| koi_steff | 0.476191 |
| koi_num_transits | 0.47397 |
| koi_kepmag | 0.399217 |
| koi_srad | 0.325253 |
| koi_rmag | 0.312893 |
| koi_smass | 0.303728 |
| koi_dikco_mdec | 0.256604 |
| koi_prad | 0.219947 |
| koi_dikco_mra | 0.15576 |
| koi_time0 | 0.115972 |
| koi_time0bk | 0.115939 |
| koi_quarters | 0.110804 |
| koi_zmag | 0.0949864 |
| dec | 0.0907086 |
| koi_fwm_sdeco | 0.0830317 |
| koi_slogg | 0.0697048 |
| koi_fwm_pdeco | 0.0402141 |
| koi_dicco_mra | -0.0142095 |
| koi_fwm_prao | -0.0591168 |
| koi_hmag | -0.0627441 |
| koi_fwm_srao | -0.109274 |
| koi_tce_plnt_num | -0.149474 |
| koi_model_snr | -0.179647 |
| koi_dicco_mdec | -0.182301 |
| koi_fwm_sdec | -0.300093 |
| koi_bin_oedp_sig | -0.315022 |
| ra | -0.459655 |
| koi_kmag | -0.640175 |
| koi_incl | -0.692183 |
| koi_count | -0.765179 |
| koi_imag | -1.39332 |
| koi_sma | -1.63119 |
| koi_fwm_stat_sig | -1.65317 |
| koi_smet | -2.21675 |
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
SHAP Importance
SHAP Dependence plots
Dependence (Fold 1)
SHAP Decision plots
Top-10 Worst decisions for class 0 (Fold 1)
Top-10 Best decisions for class 0 (Fold 1)
Top-10 Worst decisions for class 1 (Fold 1)
Top-10 Best decisions for class 1 (Fold 1)
Summary of 1_Baseline
Baseline Classifier (Baseline)
- n_jobs: -1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.4 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.652415 | nan |
| auc | 0.5 | nan |
| f1 | 0.781775 | 0.577711 |
| accuracy | 0.641732 | 0.577711 |
| precision | 0.641732 | 0.577711 |
| recall | 1 | 0.577711 |
| mcc | 0 | 0.577711 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.652415 | nan |
| auc | 0.5 | nan |
| f1 | 0.781775 | 0.577711 |
| accuracy | 0.641732 | 0.577711 |
| precision | 0.641732 | 0.577711 |
| recall | 1 | 0.577711 |
| mcc | 0 | 0.577711 |
Confusion matrix (at threshold=0.577711)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 0 | 455 |
| Labeled as FALSE POSITIVE | 0 | 815 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 5_Default_NeuralNetwork
Neural Network
- n_jobs: -1
- dense_1_size: 32
- dense_2_size: 16
- learning_rate: 0.05
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
3.1 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.203507 | nan |
| auc | 0.985399 | nan |
| f1 | 0.959756 | 0.36055 |
| accuracy | 0.948031 | 0.36055 |
| precision | 1 | 1 |
| recall | 1 | 4.77364e-13 |
| mcc | 0.886562 | 0.36055 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.203507 | nan |
| auc | 0.985399 | nan |
| f1 | 0.959756 | 0.36055 |
| accuracy | 0.948031 | 0.36055 |
| precision | 0.953939 | 0.36055 |
| recall | 0.965644 | 0.36055 |
| mcc | 0.886562 | 0.36055 |
Confusion matrix (at threshold=0.36055)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 417 | 38 |
| Labeled as FALSE POSITIVE | 28 | 787 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 4_Default_Xgboost
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: binary:logistic
- eta: 0.075
- max_depth: 6
- min_child_weight: 1
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: logloss
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
2.0 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.0853951 | nan |
| auc | 0.994887 | nan |
| f1 | 0.979268 | 0.49469 |
| accuracy | 0.973228 | 0.49469 |
| precision | 1 | 0.996172 |
| recall | 1 | 0.000171099 |
| mcc | 0.941634 | 0.49469 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.0853951 | nan |
| auc | 0.994887 | nan |
| f1 | 0.979268 | 0.49469 |
| accuracy | 0.973228 | 0.49469 |
| precision | 0.973333 | 0.49469 |
| recall | 0.985276 | 0.49469 |
| mcc | 0.941634 | 0.49469 |
Confusion matrix (at threshold=0.49469)
| Predicted as CONFIRMED | Predicted as FALSE POSITIVE | |
|---|---|---|
| Labeled as CONFIRMED | 433 | 22 |
| Labeled as FALSE POSITIVE | 12 | 803 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
